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•• The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1)^ Responsive to communication(s) filed on Amendment (05 July 05] . 
2b)M This action is FINAL. 2b)n This action is non-final. 

3) n Since this application is in condition for allowance except for fomrial matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) 13 Claim(s) 1-23 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) 13 Claim(s) 1-23 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8)n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

§)□ The specification is objected to by the Examiner. 

10) 0 The drawing(s) filed on is/are: a)n accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1. 85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action or fomn PTO-152. 

Priority under 35 U.S.C. § 119 

12) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)n All b)n Some * 0)0 None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

1 . Applicant's arguments filed 05 July 2005 have been fully considered but they are not 
persuasive. 

Applicant's Arguments 

a) Applicant states that neither Newman nor Liaguno disclose "converting the 
annotations into searchable annotations organized as hierarchical shot clusters using a voice-to- 
text conversion system." 

Examiner's Response 

b) The examiner disagrees. Newman discloses a non-linear editing system which allows 
for hypermedia (refers to the integration of text, graphics, sound, video and other data or any 
combination into a primarily associated system) for information presentation, storage and 
retrieval. Newman discloses that hypermedia environments enable users to combine elements of 
multimedia with associations to link the information. Newman discloses (Fig 9) that items are 
shown/displayed based upon source (captured fi-om 416), and the track 418. Thus given the 
broadest interpretation of the claim, the audio added is at least searchable based upon the shots 
being organized based upon the device, shot/transition/graphic/effects (402) and track. As 
stated previously, the appUcant discloses the hierarchy being related to the duration of the event, 
which is not presently claimed. 
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Claim Rejections - 35 USC §103 
2. The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this titie, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1-23 are rejected under 35 U.S. C as being unpatentable over Newman et al., US 
6,154,600 in view of Liaguno et al, US 5,729,741. 

In considering claims 1, 6, 10, 16 and 20, 
a-c) the claimed receiving a video signal is met by the graphical user interface (GUI) (Fig 14) 
where the GUI can capture a variety of sources to include, a camcorder 586, VCR 588, auxiUary 
audio TV, CD-ROM/DVD and digital camera 590. 

b) the claimed receiving a first audio signal containing annotations is met where the graphical 
user interface can capture additional audio via auxiliary audio (capture 584) and also add audio 
video via storyboard 592, which allows edit, playback and record shots, including audio shots 
(col 14, line 18-22). 

However, Newman does not specifically disclose converting the annotations into 
searchable annotations using a voice-to-text conversion system. 

Newman does disclose a system that can manipulate hypermedia, where hypermedia 
refers to the integration of text, graphics, sound, video and other data, or any combination into a 
system which can be stored and retrieved (col 5, hne 47-52). Newman also discloses that audio 
is synchronized to video by reference to a master pixel clock (col 18, Hne 57-59). Regarding the 
annotations organized as hierarchical shot clusters, Newman does disclose ranking/grouping (i.e. 
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hierarchical) the media based upon the event/scene (i.e. GreekCity, Forest, Horse, Lightening 
etc) (see Fig 9). Newman also discloses a system which lets the user input the type of 
information, i.e. "OUR SUMMER VACATION", which is displayed in the center of the window 
408 (Fig 12). Thus given the broadest interpretation of the claimed invention "hierarchical" is 
met by Newman which discloses grouping the media based upon the event/scene. 

The conversion of voice (audio, sound) to text, where the converted information is 
searchable is conventional in the art. 

The examiner incorporates Liaguno which discloses a system for storage and retrieval of 
text, pictures, video and audio where the source of information (text, pictures, video and audio) 
are converted to text. Thus Liaguno provides a system where a user is able to search (via text) 
the desired source of information, in order to provide the user rapid retrieval of any type of data 
(col 2, lines 32-42). 

Therefore, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Newman which discloses a video home editing system, which can 
edit/cut/insert/reproduce hypermedia including graphics, text and audio annotations (col 14, line 
17-19), with Liaguno, to also convert the audio to searchable text, in order to provide the user the 
ability to retrieve a desired segment, whether text, video, audio etc. . . of the hypermedia, which 
would provide the user rapid retrieval of a selected segment. 

Regarding claim 10 (the display device is met by output devices, such as TV, printers, 
computer displays etc. . . (col 4, line 26-32). 

Regarding claim 20, 

Newman discloses that decoder 204 (Fig 5) may include an optional AID converter. 
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In considering claims 2, 7, 11, 17 and 21, 

However, neither Newman nor Liaguno disclose removing the annotations from the 
second audio signal (signal 120). 

Newman discloses a media editor where the user is able to add/insert/delete/remove 
hypermedia to an existing hypermedia file by using graphical user interface 400 (Fig 9, Fig 14). 

Thus based on the type of hypermedia being edited (i.e. home video), the user could 
create a hypermedia (home video) where the scene/background audio (2°^ audio signal) is 
removed in order to provide additional/auxiliary audio in lieu of background/scene audio, thus 
removing its annotations from the second audio signal. 

Therefore it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the combination of Newman and Liaguno which discloses that hypermedia 
can be edited by deleting/removing data (video, text and audio etc), where the image data is 
converted/searchable via text, to form an edited hypermedia medium by also deleting the 
annotations of the second audio signal, in order to provide the user the ability to search the 
desired clips via annotations audio (auxihary/narrated) or the received video. 

In considering claim 3, the claimed utilizing a least-mean square algorithm 

As discussed above in claim 3, neither Newman nor Cruz disclose removing annotations 
from the second audio signal, nor removing by using a least-mean square algorithm. 

The applicant states in the specification, that the least-mean square (LMS) algorithm is a 
conventional algorithm, and also (page 12, Une 5-12) that annotations may be removed by a 
variety of methods or algorithms. 
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Therefore, it would have been obvious design choice to one of ordinary skill in the art at 
the time of the invention to modify the Newman/Liaguno combination which discloses editing 
and searching hypermedia, by utilizing an algorithm such as a LMS algorithni in removing 
annotations from the second audio signal, in order to provide an editing hypermedia which 
removes undesired audio and allows the user to insert additional/auxiliary audio if desired. 

In considering claims 4, 8, 12, 18 and 22, 1) the claimed generating a center text title,,, 
2) the claimed generating a scrolling text banner.., 

Newman discloses a system which uses a non-linear home media editor system, which 
allows the user to develop a audio/video information from either muhiple devices, to include 
recorder, VCR, TV, CD-ROM/DVD, digital camera, storyboards, intemet, etc. , , Newman also 
discloses a system which lets the user input the type of information, i.e. "OUR SUMMER 
VACATION", which is displayed in the center of the window 408 (Fig 12). Newman also 
discloses the use of scrolling text overlays, by moving prerendered graphical objects onto an 
image from a variety of hypermedia sources (col 12, line 17-23). 

In considering claims 5, 9, 13 and 19, 
The claimed generated a video abstract via the first and second audio signals, the video signals 
and the searchable annotations is met where the editing hypermedia created, generates a video 
abstract, where the audio signal(s), video signal(s) and other media which is 
included/added/inserted into the unedited hypermedia is combined to form a desired hypermedia. 
As shown in Fig 1 1, the edited hypermedia "Vacation" (444), is broken down into appropriate 
storyboards for the user which may be edited/viewed. 

In considering claim 14, 
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The claimed wherein the video signal is received from a video recorder is met by Newman 
which discloses receiving video via camcorder 586, VCR 588 and digital camera 590 (Fig 14). 

In considering claim 15, 
The claimed wherein the first and second audio signals are received from at least one 
microphone is met where Newman discloses the reception of audio/video via camcorder 586, 
VCR 588 and also auxiliary audio. 

In considering claim 23, 
The claimed wherein the video signal is received from a video recorder is met where the GUI is 
able to receive signals from camcorder 586, VCR 588 and digital camera 590 (Fig 14). 

Conclusion 

3. The prior art made of record and not rehed upon is considered pertinent to apphcant's 
disclosure — please refer to newly cited references on attached form PTO-892. 

TfflS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
pohcy as set forth in 37 CFR 1. 136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1. 136(a) will be calculated from the mailing date of the advisory action. In no event, 
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however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

4. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Brian Yenke whose telephone number is (57 1) 272-7359. The 
examiner work schedule is Monday-Thursday, 0730-1830 hrs. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
Supervisor, John W. Miller, can be reached at (571)272-7352. 
Any response to this action should be mailed to: 
Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 223 13-1450 

or faxed to: 

(571)273-8300 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the Technology Center 2600 Customer Service Office whose 
telephone number is (703)305-HELP. 

General information about patents, trademarks, products and services offered by 
the United States Patent and Trademark Office (USPTO), and other related information is 
available by contacting the USPTO's General Information Services Division at: 

800-PTO-9199 or 703-308-HELP 

(FAX) 703-305-7786 

(TDD) 703-305-7785 
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An automated message system is available 7 days a week, 24 hours a day 
providing informational responses to frequently asked questions and the ability to order 
certain documents. Customer service representatives are available to answer questions, 
send materials or connect customers with other offices of the USPTO from 8:30 am - 
8:00p.m. EST/EDT, Monday-Friday excluding federal holidays. 

For other technical patent information needs, the Patent Assistance Center can be 
reached through customer service representatives at the above numbers, Monday through 
Friday (except federal holidays) from 8:30 a.m. to 5:00 p.m EST/EDT. 

The Patent Electronic Business Center (EBC) allows USPTO customers to 
retrieve data, check the status of pending actions, and submit information and 
apphcations. The tools currently available in the Patent EBC are Patent AppUcation 
Information Retrieval (PAIR) and the Electronic Filing System (EFS). 
PAIR (http://pair.uspto.gov) provides customers direct secure access to their own patent 

application status information, as well as to general patent information publicly 
available. EFS allows customers to electronically file patent application documents 
securely via the Internet. EFS is a system for submitting new utility patent apphcations 
and pre-grant publication submissions in electronic publication-ready form. EFS 
includes software to help customers prepare submissions in extensible Markup Language 
(XML) format and to assemble the various parts of the application as an electronic 
submission package. EFS 

also allows the submission of Computer Readable Format (CRF) sequence listings for 
pending biotechnology patent applications, which were filed in paper form. 
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